I didn't implement any config-parsing related logic in it so far, marked as TODO: in the code.

Thanks.

Interesting comments from @ericspod during the online meeting:

1. Just a note that .ts is meant to be Torchscript but to web developers it'll looks like a Typescript file, we may want to choose something else if anyone finds that confusing.
2. Versioning the package would help, if we used git repos to store things the commit hash becomes part of the version implicitly.

Thanks.

Hi @ericspod @wyli ,

Maybe I misunderstood your suggestion, how can we get the commit hash if we want to save the commit hash in meta data for this commit...?

Thanks in advance.

Hi @ericspod @wyli ,

Maybe I misunderstood your suggestion, how can we get the commit hash if we want to save the commit hash in meta data for this commit...?

Thanks in advance.

What I was thinking was, from the perspective of a user, the commit hash could be used to determine model versions. It wouldn't be included in the package but would be used to tell model versions apart that claim to be the same version, ie. if the user forgets to increment the version.

Hi @ericspod @wyli ,
Maybe I misunderstood your suggestion, how can we get the commit hash if we want to save the commit hash in meta data for this commit...?
Thanks in advance.

What I was thinking was, from the perspective of a user, the commit hash could be used to determine model versions. It wouldn't be included in the package but would be used to tell model versions apart that claim to be the same version, ie. if the user forgets to increment the version.

Thanks for your explanation, makes sense to me!

Slightly adjusted the folder structurer, thanks for the reference of MONAI application:
https://github.com/Project-MONAI/monai-deploy/blob/main/guidelines/monai-application-package.md#table-of-important-paths

Thanks.

Hi @wyli @ericspod ,

Thanks for several internal discussions, I updated the example model package to show new ideas:

1. inference.py shows how to get config and modify it then instantiation, and also shows how to get instance directly, these are the 2 ways for JSON / python hybrid.
2. inference_v2.py shows how to write a new config with a base config and override some configs.
3. trtinfer.json and trtinfer.py show how to refer to config in another JSON file or same file, simulated TensorRT and DALI inference logic.

Could you please help take a look again?

Thanks in advance.

Hi @ericspod @wyli ,

I added the tensorboard.png and mlflow.png into the model package for training curves.

Thanks.

Hi @aihsani ,

Thanks for your suggestion!
I think the shell scripts are used to simplify the common use cases and args, especially for researchers to tune args when training a MMAR, for example, they may have several different configs, change config file in shell.
For example, the inference.sh in this example is:

python ../scripts/inference.py
    --config ../configs/inference.json
    --meta ../configs/metadata.json

And you are right, we should definitely not put complicated logic in the shell script, main things should in the python or JSON files, you can call the python program inference.py directly.

Thanks.

I added schema example to check the metadata format and expected values.
It's based on the jsonschema standard: http://json-schema.org/.
I didn't define schema for all the items, just included key required items as example.
We can define MONAI MMAR schema and put them into some public web storage in the future.

Thanks.

Hi @wyli ,

I changed the override feature to a separate MMAR example according to our online discussion to make it more clear.

Thanks.

Hi @ericspod @wyli @rijobro @aihsani ,

I have updated this MMAR example according to our online discussion yesterday.
And moved the python scripts and schema to MONAI core to support 90% cases:
https://github.com/Project-MONAI/MONAI/tree/506b9089637d7dd9ed3496dde9deee8c669f269d/monai/apps/mmars

Thanks.

To recap what we had discussed about actual file structure, let's say we have a directory with at least these files:

models/model.ts
models/model.pt
configs/metadata.json

We can consider this directory the model package itself. The metadata.json would have in it the information that is currently provided in this PR, plus it should also have the arguments to instantiate the network rather than that being in the inference config. If the model isn't compatible with Torchscript then the mode.ts file will be absent and this will be needed to recreate the network before loading the stored weights.

This directory can then be packaged into a zip file whose structure can be assumed with models and configs directories. If the model is compatible with Torchscript then a new .ts file can be made with the metadata stored internally and other files saved as extra files using save_net_with_metadata. Users can then load this Torchscript model without worrying about the metadata or anything else MONAI specific if they just wanted to get things started quickly. Either way we'd have a zip file that's easy to distribute and which other software can expect to have as input.

closing in favour of https://github.com/Project-MONAI/tutorials/tree/master/modules/bundles